Unsupervised image registration commonly adopts U-Net style networks to predict dense displacement fields in the full-resolution spatial domain. For high-resolution volumetric image data, this process is however resource intensive and time-consuming. To tackle this problem, we propose the Fourier-Net, replacing the expansive path in a U-Net style network with a parameter-free model-driven decoder. Specifically, instead of our Fourier-Net learning to output a full-resolution displacement field in the spatial domain, we learn its low-dimensional representation in a band-limited Fourier domain. This representation is then decoded by our devised model-driven decoder (consisting of a zero padding layer and an inverse discrete Fourier transform layer) to the dense, full-resolution displacement field in the spatial domain. These changes allow our unsupervised Fourier-Net to contain fewer parameters and computational operations, resulting in faster inference speeds. Fourier-Net is then evaluated on two public 3D brain datasets against various state-of-the-art approaches. For example, when compared to a recent transformer-based method, i.e., TransMorph, our Fourier-Net, only using 0.22$\%$ of its parameters and 6.66$\%$ of the mult-adds, achieves a 0.6\% higher Dice score and an 11.48$\times$ faster inference speed. Code is available at \url{https://github.com/xi-jia/Fourier-Net}.

由于其极端的长距离建模能力，基于视觉变压器的网络在可变形图像注册中变得越来越流行。但是，我们认为，5层卷积U-NET的接受场足以捕获准确的变形而无需长期依赖性。因此，这项研究的目的是研究与现代变压器的方法相比，将基于U-NET的方法用于医学图像注册时是否已过时。为此，我们通过将平行的卷积块嵌入香草U-NET以增强有效的接受场来提出一个大核U-NET（LKU-NET）。在公共3D IXI Brain Dataset上，用于基于ATLAS的注册，我们表明，香草U-NET的性能已经与基于最新的变压器网络（例如Transmorph）相提并论，并且提出的LKU-NET仅使用其参数的1.12％和其多添加操作的10.8％，优于Transmorph。我们进一步评估了MICCAI Learn2Reg 2021挑战数据集中的LKU-NET，以进行主题间注册，我们的LKU-NET在此数据集中也优于TransMorph，并且在此工作提交后，在公共排行榜上排名第一。只有对香草U-NET的适度修改，我们表明U-NET可以在基于主体间和基于ATLAS的3D医疗图像注册上胜过基于变压器的体系结构。代码可在https://github.com/xi-jia/lku-net上找到。

随着深度学习的兴起，视频对象细分（VOS）取得了重大进展。但是，仍然存在一些棘手的问题，例如，类似的对象很容易混淆，很难找到微小的对象。为了解决这些问题并进一步提高VOS的性能，我们为这项任务提出了一个简单而有效的解决方案。在解决方案中，我们首先分析YouTube-VOS数据集的分布，并通过引入公共静态和视频分割数据集来补充数据集。然后，我们改善了具有不同特征的三个网络体系结构，并训练多个网络以学习视频中对象的不同特征。之后，我们使用一种简单的方法来集成所有结果，以确保不同的模型相互补充。最后，进行了微妙的后处理，以确保具有精确边界的准确视频对象分割。 YouTube-VOS数据集的大量实验表明，该建议的解决方案在YouTube-VOS 2022测试集上以86.1％的总分达到了最先进的性能，这是YouTube视频对象细分的第五名-VOS挑战2022。

在图像识别中已广泛提出了生成模型，以生成更多图像，其中分布与真实图像相似。它通常会引入一个歧视网络，以区分真实数据与生成的数据。这样的模型利用了一个歧视网络，该网络负责以区分样式从目标数据集中包含的数据传输的数据。但是，这样做的网络着重于强度分布的差异，并可能忽略数据集之间的结构差异。在本文中，我们制定了一个新的图像到图像翻译问题，以确保生成的图像的结构类似于目标数据集中的图像。我们提出了一个简单但功能强大的结构不稳定的对抗（SUA）网络，该网络在执行图像分割时介绍了训练和测试集之间的强度和结构差异。它由空间变换块组成，然后是强度分布渲染模块。提出了空间变换块来减少两个图像之间的结构缝隙，还产生了一个反变形字段，以使最终的分段图像背部扭曲。然后，强度分布渲染模块将变形结构呈现到具有目标强度分布的图像。实验结果表明，所提出的SUA方法具有在多个数据集之间传递强度分布和结构含量的能力。

Non-line-of-sight (NLOS) imaging aims to reconstruct the three-dimensional hidden scenes from the data measured in the line-of-sight, which uses photon time-of-flight information encoded in light after multiple diffuse reflections. The under-sampled scanning data can facilitate fast imaging. However, the resulting reconstruction problem becomes a serious ill-posed inverse problem, the solution of which is of high possibility to be degraded due to noises and distortions. In this paper, we propose two novel NLOS reconstruction models based on curvature regularization, i.e., the object-domain curvature regularization model and the dual (i.e., signal and object)-domain curvature regularization model. Fast numerical optimization algorithms are developed relying on the alternating direction method of multipliers (ADMM) with the backtracking stepsize rule, which are further accelerated by GPU implementation. We evaluate the proposed algorithms on both synthetic and real datasets, which achieve state-of-the-art performance, especially in the compressed sensing setting. All our codes and data are available at https://github.com/Duanlab123/CurvNLOS.

In this paper, we target at the problem of learning a generalizable dynamic radiance field from monocular videos. Different from most existing NeRF methods that are based on multiple views, monocular videos only contain one view at each timestamp, thereby suffering from ambiguity along the view direction in estimating point features and scene flows. Previous studies such as DynNeRF disambiguate point features by positional encoding, which is not transferable and severely limits the generalization ability. As a result, these methods have to train one independent model for each scene and suffer from heavy computational costs when applying to increasing monocular videos in real-world applications. To address this, We propose MonoNeRF to simultaneously learn point features and scene flows with point trajectory and feature correspondence constraints across frames. More specifically, we learn an implicit velocity field to estimate point trajectory from temporal features with Neural ODE, which is followed by a flow-based feature aggregation module to obtain spatial features along the point trajectory. We jointly optimize temporal and spatial features by training the network in an end-to-end manner. Experiments show that our MonoNeRF is able to learn from multiple scenes and support new applications such as scene editing, unseen frame synthesis, and fast novel scene adaptation.

In this paper, we propose a large-scale language pre-training for text GENeration using dIffusion modEl, which is named GENIE. GENIE is a pre-training sequence-to-sequence text generation model which combines Transformer and diffusion. The diffusion model accepts the latent information from the encoder, which is used to guide the denoising of the current time step. After multiple such denoise iterations, the diffusion model can restore the Gaussian noise to the diverse output text which is controlled by the input text. Moreover, such architecture design also allows us to adopt large scale pre-training on the GENIE. We propose a novel pre-training method named continuous paragraph denoise based on the characteristics of the diffusion model. Extensive experiments on the XSum, CNN/DailyMail, and Gigaword benchmarks shows that GENIE can achieves comparable performance with various strong baselines, especially after pre-training, the generation quality of GENIE is greatly improved. We have also conduct a lot of experiments on the generation diversity and parameter impact of GENIE. The code for GENIE will be made publicly available.

Structured tabular data exist across nearly all fields. Reasoning task over these data aims to answer questions or determine the truthiness of hypothesis sentences by understanding the semantic meaning of a table. While previous works have devoted significant efforts to the tabular reasoning task, they always assume there are sufficient labeled data. However, constructing reasoning samples over tables (and related text) is labor-intensive, especially when the reasoning process is complex. When labeled data is insufficient, the performance of models will suffer an unendurable decline. In this paper, we propose a unified framework for unsupervised complex tabular reasoning (UCTR), which generates sufficient and diverse synthetic data with complex logic for tabular reasoning tasks, assuming no human-annotated data at all. We first utilize a random sampling strategy to collect diverse programs of different types and execute them on tables based on a "Program-Executor" module. To bridge the gap between the programs and natural language sentences, we design a powerful "NL-Generator" module to generate natural language sentences with complex logic from these programs. Since a table often occurs with its surrounding texts, we further propose novel "Table-to-Text" and "Text-to-Table" operators to handle joint table-text reasoning scenarios. This way, we can adequately exploit the unlabeled table resources to obtain a well-performed reasoning model under an unsupervised setting. Our experiments cover different tasks (question answering and fact verification) and different domains (general and specific), showing that our unsupervised methods can achieve at most 93% performance compared to supervised models. We also find that it can substantially boost the supervised performance in low-resourced domains as a data augmentation technique. Our code is available at https://github.com/leezythu/UCTR.

Making sense of multiple modalities can yield a more comprehensive description of real-world phenomena. However, learning the co-representation of diverse modalities is still a long-standing endeavor in emerging machine learning applications and research. Previous generative approaches for multimodal input approximate a joint-modality posterior by uni-modality posteriors as product-of-experts (PoE) or mixture-of-experts (MoE). We argue that these approximations lead to a defective bound for the optimization process and loss of semantic connection among modalities. This paper presents a novel variational method on sets called the Set Multimodal VAE (SMVAE) for learning a multimodal latent space while handling the missing modality problem. By modeling the joint-modality posterior distribution directly, the proposed SMVAE learns to exchange information between multiple modalities and compensate for the drawbacks caused by factorization. In public datasets of various domains, the experimental results demonstrate that the proposed method is applicable to order-agnostic cross-modal generation while achieving outstanding performance compared to the state-of-the-art multimodal methods. The source code for our method is available online https://anonymous.4open.science/r/SMVAE-9B3C/.

The dual-encoder has become the de facto architecture for dense retrieval. Typically, it computes the latent representations of the query and document independently, thus failing to fully capture the interactions between the query and document. To alleviate this, recent work expects to get query-informed representations of documents. During training, it expands the document with a real query, while replacing the real query with a generated pseudo query at inference. This discrepancy between training and inference makes the dense retrieval model pay more attention to the query information but ignore the document when computing the document representation. As a result, it even performs worse than the vanilla dense retrieval model, since its performance depends heavily on the relevance between the generated queries and the real query. In this paper, we propose a curriculum sampling strategy, which also resorts to the pseudo query at training and gradually increases the relevance of the generated query to the real query. In this way, the retrieval model can learn to extend its attention from the document only to both the document and query, hence getting high-quality query-informed document representations. Experimental results on several passage retrieval datasets show that our approach outperforms the previous dense retrieval methods1.